Kruiser: Semi-synchronized Non-blocking Concurrent Kernel Heap Buffer Overflow Monitoring

Kernel heap buffer overflow vulnerabilities have been exposed for decades, but there is still no practical countermeasure that can be applied to the OS kernel. Previous solutions either suffer from high performance overhead or compatibility problems with the existing kernel and hardware. In this paper, we present Kruiser, a concurrent kernel heap buffer overflow monitor. Unlike conventional methods, the security enforcement of which are usually inlined into the kernel execution, we introduce a concurrent monitor process, which decouples security mechanisms from the kernel’s normal execution, leveraging the increasingly popular multicore architectures. To reduce the synchronization overhead between the monitor process and the running kernel, we design a novel semi-synchronized non-blocking monitoring algorithm, which enables an efficient runtime detection on live memory without incurring false positives. To prevent the monitor process from being tampered and provide guaranteed performance isolation, we utilize the virtualization technology to run the monitor in a trusted environment without affecting performance. We have implemented a prototype of Kruiser based on Linux and the Xen hypervisor. The evaluation shows that Kruiser can detect realistic kernel heap buffer overflow attacks effectively with minimal overhead. It imposes 2.7% throughput reduction on Apache and negligible performance overhead on SPEC CPU2006.